Skip to content

fix(pi): narrow converter guards — trailing assistant, tool/image/surrogate, signed-thinking#32

Open
iceteaSA wants to merge 2 commits into
cortexkit:mainfrom
iceteaSA:fix/pi-improvements
Open

fix(pi): narrow converter guards — trailing assistant, tool/image/surrogate, signed-thinking#32
iceteaSA wants to merge 2 commits into
cortexkit:mainfrom
iceteaSA:fix/pi-improvements

Conversation

@iceteaSA
Copy link
Copy Markdown
Contributor

@iceteaSA iceteaSA commented May 21, 2026

Narrow, Pi-owned converter hardening for packages/pi/src/convert.ts, beyond what was merged in #17. All changes are scoped to concrete invalid shapes the Pi converter can produce — no broad payload "healing".

Commit 1: strip trailing assistant messages, improve sanitize and error guards

  • Strip trailing assistant messages from converted requests to prevent Anthropic prefill 400s on models that reject assistant-message prefill.
  • Unicode regex (/gu flag) for spec-compliant surrogate-pair replacement.
  • Simplified sanitizeToolId with a cleaner length check (removes a redundant guard).
  • Improved error-content guard catches the empty-string case (content === '') that upstream misses.
  • Comprehensive test coverage for the conversion logic.

Commit 2: preserve signed thinking verbatim, guard lone surrogates

  • Send signed thinking blocks byte-identical. The signature is computed over the original text, so sanitizing would alter it and Anthropic rejects the block as modified. Previously the converter sanitized signed thinking unconditionally; now it preserves it verbatim (the normal path for Anthropic-origin reasoning).
  • Lone-surrogate exception only. A signed block containing a lone (unpaired) UTF-16 surrogate can't keep its signature — sanitizing breaks it, and the raw surrogate is invalid UTF-8 (400). hasLoneSurrogate() detects exactly that case, and only then is the signature dropped and the text sanitized.
  • Tests cover verbatim signed-thinking preservation and the lone-surrogate downgrade.

Scope note (per review)

This PR intentionally does not include the broad "downgrade all historical signed thinking to text" healing. That approach can avoid 400s but discards valid signed/redacted reasoning when the request shape is otherwise correct. The signed-thinking reordering problem is better fixed at the OpenCode layer in anomalyco/opencode#30182 (preventing the Anthropic reorder from displacing signed/redacted reasoning in the first place). For Pi, the converter should avoid broad destructive healing unless a concrete Pi-generated invalid shape requires it — none currently does, so it is omitted here.

Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 3 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread packages/pi/src/tests/convert.test.ts
@iceteaSA iceteaSA force-pushed the fix/pi-improvements branch from b659ab4 to 768e897 Compare May 22, 2026 17:08
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Tip: cubic could auto-approve low-risk PRs like this, if it thinks it's safe to merge. Learn more

Re-trigger cubic

@iceteaSA iceteaSA force-pushed the fix/pi-improvements branch from 768e897 to 432f588 Compare May 29, 2026 17:28
@ualtinok
Copy link
Copy Markdown
Contributor

ualtinok commented Jun 1, 2026

Reviewed this in context with OpenCode PR anomalyco/opencode#30182 and the related OpenCode-side healing PRs here (#49, #51).

My read:

  • The Pi conversion tests and the narrower conversion hardening are useful: trailing assistant stripping, empty is_error tool result handling, tool ID/image/surrogate guards are all reasonable Pi-owned converter fixes.
  • The historical signed-thinking downgrade should not be merged as-is. It is the same broad “heal the final payload by converting signed thinking to text” approach as fix(opencode): repair thinking blocks rejected by Anthropic API #49. That can avoid Anthropic 400s, but it also discards valid signed/redacted thinking that should be preserved when the converter/request shape is correct.
  • For OpenCode, PR #30182 is the better layer for the signed-thinking issue because it prevents OpenCode’s Anthropic reorder from moving signed/redacted reasoning in the first place. For Pi, OpenCode cannot fix the converter, but the Pi converter should still avoid broad destructive healing unless a concrete Pi-generated invalid shape requires it.

Recommendation: split this PR. Keep the tests and the narrow Pi conversion guards, including trailing assistant stripping. Drop or separately rework the historical signed-thinking downgrade into a much narrower fix backed by a concrete Pi repro.

@iceteaSA iceteaSA force-pushed the fix/pi-improvements branch 2 times, most recently from c3d0d62 to 0003da7 Compare June 1, 2026 08:03
@iceteaSA iceteaSA changed the title fix(pi): strip trailing assistant messages, improve sanitize and error guards fix(pi): narrow converter guards — trailing assistant, tool/image/surrogate, signed-thinking Jun 1, 2026
@iceteaSA
Copy link
Copy Markdown
Contributor Author

iceteaSA commented Jun 1, 2026

Recommendation: split this PR. Keep the tests and the narrow Pi conversion guards, including trailing assistant stripping. Drop or separately rework the historical signed-thinking downgrade into a much narrower fix backed by a concrete Pi repro.

Dropped it

iceteaSA added 2 commits June 1, 2026 16:44
…r guards

- Strip trailing assistant-role messages from converted requests to prevent
  Anthropic prefill 400 errors on some models
- Add Unicode flag to surrogate regex for spec compliance
- Simplify sanitizeToolId with cleaner length check
- Improve error content guard to catch empty string case
- Add comprehensive test coverage for all pi conversion logic
Signed thinking blocks must be sent back byte-identical — the signature is
computed over the original text, so sanitizing would alter it and Anthropic
rejects the block as modified. Send signed thinking verbatim instead of
sanitizing it.

The one exception is a signed block containing a lone (unpaired) UTF-16
surrogate: the signature cannot survive sanitization and the raw surrogate is
invalid UTF-8 (400). Detect lone surrogates with hasLoneSurrogate() and, only
in that concrete case, drop the signature and downgrade to sanitized text.

No broad historical-thinking healing — valid signed/redacted thinking is
preserved. (Signed-thinking reordering is handled OpenCode-side in #30182.)
@iceteaSA iceteaSA force-pushed the fix/pi-improvements branch from 0003da7 to ae0626e Compare June 1, 2026 14:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants